Range sensing has important industrial applications. Examples of processes which benefit greatly from the use of range imagery include: the measurement of solder paste volume in manufacture of surface-mounted electronic assemblies; digitization of the geometry of three-dimensional clay models; and inspection of semiconductor packages for lead coplanarity.
Stereo vision, triangulation, time-of-flight, and structured lighting methods are all used to obtain depth images. All of these methods require computational overhead which constrains the rate at which dense depth images can be produced. Another class of depth imaging techniques relies on the focus information present in camera images due to the depth of field constraints of optical systems. Depth from focus methods employ a search for the focal position yielding best camera focus on a point-by-point basis. A focus measure, usually a form of high pass filter, is used to determine the focus maxima, as described in Subbaro, M. and Gopal, S., Depth from Defocus: A Spatial Domain Approach, International Journal of Computer Vision, 13, 3, 271-294, 1994, incorporated herein by reference. Depth in the observed scene can be determined from focal position using basic lens equations. The technique benefits from the fact that all spatial frequencies present in an image attain peak amplitude at best focus. Depth from focus suffers from the drawback that many images need to be taken to locate the focal maxima with sufficient accuracy.
Pentland, in "A New Sense For Depth of Field", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 9, No. 4, pp. 523-531, July 1987, incorporated herein by reference, demonstrated the use of focal gradient information to derive depth images from one or two images of a scene, without requiring a more exhaustive search for point-by-point maxima. This technique has become known as depth from defocus. In passive depth from defocus, the high frequency information naturally present in a scene is analyzed to determine focal position. However, this natural object texture is lacking when viewing smooth objects. Pentland and Girod suggested the use of structured illumination to circumvent this problem, in A. Pentland, S. Scherock, T. Darrell, and B. Girod, Simple Range Cameras Based on Focal Error, J. Optical Society of America, vol. 11, no. 11, pp. 2925-2935, November 1994, and B. Girod and S. Scherock, Depth from Focus of Structured Light, Proc. SPIE: Optics, Illum., and Image Sng for Mach Vis. IV, vol. 1194, November 1989, Philadelphia, Pa. herein incorporated by reference. A fine two-dimensional light pattern is used to (actively) illuminate the object, creating artificial texture. Defocus of the superimposed illumination pattern conveys range information.
Note that the term texture is used here to describe variations in reflectance seen in two dimensional images. This texture may or may not be due to actual surface relief.
An important simplification results from the use of active illumination in depth from defocus. Many different spatial frequencies are present in natural scenes. These different signal components are attenuated differently by defocus. Deriving a focus measure which is invariant to spatial frequency has thus been difficult. Nayer et. al recognized that imposition of structured illumination permits control over the spectral content of images., as described in S. Nayer, M. Watanabe, M. Noguchi, Real-time Focus Range Sensor, IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 18, No. 12, pp. 1186-1198, December 1996, herein incorporated by reference. An active illumination pattern was developed in conjunction with a narrow band tuned focus measure. The resulting range sensor was demonstrated to produce range images of good quality in real-time, made possible by the computational simplification of the focus measure.
In active depth from defocus using a tuned focus measure, it is necessary that the projected structured illumination dominate the texture observed in the object. Natural object texture present within the passband of the tuned operator is not rejected. Furthermore, very high contrast regions in the reflectance image, such as reflections from the tops of solder balls on integrated circuit packages, can induce Gibbs ringing in the narrow-band tuned focus measure. These effects create noise and artifacts in the range images. Real objects of interest in machine vision applications contain mixtures of materials with different spatial frequency content and amplitude of natural texture. Also, at the scale of semiconductor packaging, translucency of surfaces blurs the projected pattern and greatly reduces it's contrast. Thus, we find that projected texture does not always dominate observed texture, as required.